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NON-ZERO-SUM STOPPING GAMES IN CONTINUOUS TIME 


ZHOU ZHOU 


Abstract. On a filtered probability space (f2,T, (J r t)tG[0,oo]>lP)> we consider the two-player non- 
zero-sum stopping game u* := E[[7* (p,r)], i = 1,2, where the first player choose a stopping strategy 
p to maximize u * 1 and the second player chose a stopping strategy r to maximize u 2 . Unlike the 
Dynkin game, here we assume that U(s,t) is J^vt-measurable. Assuming the continuity of U l in 
(s,t), we show that there exists an e-Nash equilibrium for any e > 0. 


1. Introduction 

On a filtered probability space (0, J 7 , (-Tt)ig[o,oo]i IP)) we consider the two-player non-zero-sum 
stopping game u l (p,r) := E[U l (p, r)], i = 1,2, where the first player choose a stopping strategy 
p to maximize u 1 and the second player chose a stopping strategy r to maximize u 2 . Unlike the 
Dynkin game, here we assume that U(s,t ) is J^vt-measurable instead of J^At-measurable. That is, 
the game ends at the maximum of p and r. Moreover, p and r are not stopping times. They are 
some strategies satisfying some non-anticipativity condition. By assuming the continuity of U l in 
(s,t), we show that there exists an e-Nash equilibrium for any e > 0. 

We first provide some results in the zero-sum case. In particular, we construct an e-saddle point. 
(A more general version for the zero-sum case has been studied in [1].) Then we construct an e-Nash 
equilibrium of the non-zero-sum game by using the e-saddle points in the zero-sum case. 

To avoid the technical difficulties stemming from the verification of the path regularity of some 
related processes, we assume that the state space is at most countable. Besides, we assume that 
the payoff function U l is uniformly continuous in (t,s). These assumptions, though restricted, help 
us concentrate on the interesting parts of the paper. It can be expected that the result still holds 
in a more general framework (The right continuity of U l is always in force). 

Compared to the non-zero-sum Dynkin game (see e.g., i» our game is more applicable. In 
practice, even if a player has made the decision first, her payoff can still be affected by other players’ 
decisions later on. Therefore, it is more reasonable to let the game end at the maximum rather 
than the minimum of the stopping strategies. As far as we know, this is the first paper studying 
the stopping games with this feature in a non-zero-sum setup. (Recently there are two papers pL][2] 
on the stopping games with this feature in the zero-sum case.) 

The paper is organized as follows. In the next section, we introduce the setup and the main 
result. In Section 3, we provide some results in the zero-sum case. In Section 4, we use the results 
from the zero-sum case to construct an e-Nash equilibrium for the non-zero-sum stopping games. 
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2. The setup and the main result 

Let (12, J 7 , (-Tt)te[o,cxDl)JF > ) be a filtered probability space, where without loss of generality T = 
Too = U te[ o i00 )Ji, and the filtration (J’t)te[ 0 ,oo] satisfies the usual conditions. To avoid the technical 
difficulties stemming from the verification of the path regularities of some related related processes, 
we assume that 12 is at most countable, and that P is supported on 17. Let T be the set of stopping 
times taking values in [0,oo]. For any a £ T, denote T a (resp. T a +) as the set of stopping times 
that is no less (resp. strictly greater) than a. Let 

T := {4> : [0, oo] x 12 i-a [0, oo] : is £>([0, oo)) 0 T — measurable, and Vi £ [0, oo), <f{t, •) £ %+}■ 

Definition 2.1. p = (po,pi) is said to be a stopping strategy, if po £ T and p\ £ T. Denote the 
set of stopping strategies as T. 

For a £ T and £ T, denote 4>{cr) := 4>(a(-), •). For any p = (po,pi),T = (to,ti) £ T, denote 

P[ T } ■ A)1{po<to} ~b Pl( T o)l{po>ro}- 

For i = 1,2, let U l : [0, oo) x [0, oo) x 17 i-a R, such that U(s,t ) is J^vt-measurable. For simplicity, 
we assume that U l is bounded, and for any s,t £ [0, oo), the limits 

U l {s, oo) := inf U l (s,b), U l (oo,t) := inf U l {a,t ), U l { oo,oo):= inf U l {a,b) 

6—>• cxd a—>■ oo a,b—> oo 

exist, i = 1,2. We make the following assumption on U l . 

Assumption 2.2. There exists a bounded non-decreasing function r : [0, oo] t-> M + with lim^o r(h) = 
r(0) = 0, and 

\U l (s,t,-) — [T(s' ,t' , -)| < r (|s — s'| + |i — i'l) , Vs, t, s', t' £ [0, oo], i = 1,2. 

Consider the two-player non-zero-sum stopping game 

u\p,r) = E [U 1 {p[t\,t[p})\ (2.1) 

E \jJ (po, (po)) l{po<m} U \p\ (to), To)l{p 0 >T 0 } + U (po, po)l{po=ro}] i 

p = (poi Pi), t = (to, ti) £ T, i = 1,2. Here the first player choose p to maximize u 1 and the second 
player choose r to maximize u 2 . 

Recall the definition of an e-Nash equilibrium. 

Definition 2.3. For e > 0, (p*,r*) £ T 2 is said to be an e-Nash equilibrium for the stopping game 
(12.11) . if for any p,r £ T, 

u l (p,T*) < u l (p*, t*) + e and u 2 (p* , t) < u 2 (p* , r*) + e. 

Below is the main result of this paper. 

Theorem 2.4. Under Assumption ^. 21 there exists an e-Nash equilibrium for (12.11) for any e > 0. 
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3. Zero-sum case 

In this section, we consider (|2.1j) in the zero-sum case, i.e., we assume U 1 = —U 2 := U throughout 
this section. For t £ [0, oo], let 

X t := inf E t [U(t,a)], Y t := sup E t [U(a,t)] and Z t := U(t,t), 

<y&Tt a£ Tt 

where E CT [-] := E[-|Jv] for any cr £ T- Obviously, 

X t <Z t <Y t . (3.1) 


Lemma 3.1. The processes (^t)te[0,oo] an d (^t)te[0,oo] are right continuous. 
Proof. Fix t £ [0, oo). Let t n \t with | t n — t\ < 1/n. Let 

X v := ME v [U(t,a)], v £ %■ 

creTv 

Then Xt = Xt. We have that 


limsup \X tn - X t \< limsup \X tn - X tn + lirnsup \X t „ ~ X t 


n—> oo 


n—>oo 

n—too 

limsup \X tn -X tn \ 

_ 

lim sup 

inf E t [U(t n , a)] — inf E tn [U(t, cr)] 

n—>• oo 


n—>oo 

°£T tn aeT tn 


< 

lim sup 

n—>oo 

sup E in \U(t n , a ) - U{t,a)\ 

7&Tt n 


< 

lim sup 

n—>oo 

0 . 

r(l/n) 


By [3 Theorem D.7], limsr^^^ \X tn — X t \ =0. Hence, the result follows. 

Let 

V {PoXo) ■— E \_Xp 0 l{po<''"o} "B ^ro^-{po>T'o} "B ^Pq^-{po=to}\ , P, T B 
and consider the zero-sum Dynkin game 

v := sup inf V{pq,tq) and v := inf sup V(po, To). 
PoeT T oeT r 0 gT Po er 


□ 


(3.2) 


By (|3.1I) and Lemma [3TT1 and the game has a value and there exists an e-saddle point (pg, Tq) £ T 2 
for any e > 0,. That is, v = v and for any po, tq £ T, 

v{po, To) - e < V ( Po , Tq) < V(Po,T 0 ) + e. 

Now let h > 0. For each n £ N, let p(nh) £ T n h be the e-optimizer for Y n h. That is, 

E n fe [U(p(nh), nh)] > sup E [U(a,nh)\ — e. 

G^T'nh 


Define ph : [0, oo] x O ia [0, oo], 


Ph{t) ■= p({[t/h\ + !) h) , t £ [0, oo), 


(3.3) 
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and ph{ oo) = oo. Then it is easy to see that ph £ T. Similarly, let r(nh) be the e-optimizer for 
X n h, and define Th : [0,oo] x [0,oo], 


T h {t) := t {{[t/h\ + 1) h) , f<G[0, oo), 


(3.4) 


and Th{ oo) = oo. Observe that ph(-,uj) and Th(-,ca) are right continuous for each u £ Q. This right 
continuity will play an important role in the proof of Theorem 12.41 in the next section. 

Lemma 3.2. For any e > 0, there exists h! > 0, such that for any h £ (0, h'), 


X po > E po [U(po,T h (p 0 ))\ - 2e and Y TQ < E T0 [U(p h (T 0 ),T 0 )] + 2e, Vp 0 ,r 0 6 T, (3.5) 
Proof. For tq £ T, denote r 0 h = (bo /h] + 1 )h. Then 


Y to -E ro [U{p h (T 0 ),T 0 )}\ 


< 


< 


< 


< 


sup E ro [U(a,T 0 )\ - E ro [U(p(T 0h ),T 0h )] 

r£Tr 0 

sup E ro [U(a, ro)] - E ro [E roh [U(p{r oh ), T 0h )]] 

r&Tr Q 

sup E ro [U(a,r 0 )\ - E ro 

r£Tr 0 

sup E ro [U(a,r Q )\ - E ro 

r£Tr 0 

sup E to [U(a,To)\ - sup E ro [U(a, t 0 )] 


sup E r 0h [U{a,T Qh )\ 


sup E r oh [U{<J,T 0 )} 


o-er^o 


re7 U)h 


sup E ro [U(a,T 0 )\ - sup E ro [U(a V T 0 ft,To)] 

ct&Tt q 

< sup E ro \U(a,To) - U{a Vr ft ,r 0 )| +e + 2r(h) 

a&Tr 0 

< 3 r(h) + e. 


+ I E r 0 [U(p(T 0h ),T 0h )] -E tq [U(p(T 0h ),T 0 )]\ 
+ r(h) 

+ e + r(h) 

+ e + 2 r(h) 

+ e + 2 r(h) 

+ e + 2r(h) 


Now choose h! > 0, such that r(h') < e/3. Then the result follows. □ 

Proposition 3.3. For any e > 0, let p e = (pfj, Ph) and r e = (r/, t^), where (/c>o; r o) a saddle point 
for (|3. 2 p . and h zs small enouyh such that <ra holds. Then (p e ,r e ) is a he-Nash equilibrium for 
m when U 1 = — U 2 = U. Flence Theorem \2.J\ holds in the zero-sum case. 
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Proof. Let p = {po,pi) £ T. We have that 

u l (p,T e ) = E U(p 0 ,T h (p 0 ))l {po<T e } + U(p 1 {T^),T^)l {po>T e } + U(p 0 ,p 0 )l {po=TS} 

- E Ep 0 [U(pO, T/j(po))] 1{po<Tq} T E r e [U (pi (Tq), Tq )] 1{p 0 >Tq} + / 5 o)l{po=Tg} 

< E Xp 0 l{p 0>T £| + l r el{p 0<r £} + Zp 0 l{p 0=T e}. + 2e 


< E 

< E 
= u 1 ( / 9 e ,T e ) + 5e 

Similarly, we can show that 


^Po'*‘{Po >r o} ■^ r d'*‘{Po <r ol ^d^{Pd= T dl 


+ 3e 


K Pb [' U(po,Th(Po))\ l{p§<T 0 '} +E r « [U(ph(T^),T^)\ l{ P 5>r 0 £ } + U(p e 0l Po)l {p c o=T ‘ } 


+ 5e 


u 2 (p € ,t) < u z (p e ,r £ ) + 5e. 


2 ( e f\ 


□ 


Remark 3.4. We can also consider the sub-game of (12.11) 


r(p,r) =E a [U 1 (p[t],t\p])\ , (p,r) £ T 2 , ct<eT, i = 1,2, 


(3.6) 


where 


“La := {(po,/h) € T : > cr}- 

Follow the same proof as above, we can show that ((p%,Ph), (r^, ph )) £ is a 5 e-Nash equilibrium 
for (13.61) when U 1 = — U 2 = U, where (p%,T £ ) is an e-saddle point for the sub-Dynkin game 

v a sup inf Eg- \fX po l{p 0 < T0 ) 3" ^ to l{po>ro} T ^po^{po=to}] 

P&Ta TGT <t 

~ (iff SU P [^PO^-{po<To) ^ r O^{po>To} “I" Zp 0 l{po=ro}] • 

Tg Ft p^Ta 

Moreover, as can be seen from the proof of Proposition HOI for any (po,pi) £ 

ul((po, Pi), K,r h )) <v a + 4e. 

4. Proof of Theorem 12.41 
Throughout this section, we fix e > 0. For t £ [0, oo], let 


and 


X} := inf Et [U\t , a)] , Y t l := sup E f [u\a, i)] , Z\ = U\t , t), 

a&Tt 


X 2 := sup Et [U 2 (t, a)] , T t 2 := inf E* [U 2 (a, t)] , Z 2 = U 2 (t , t). 
a£Tt 


Similar to (13.31) and (13.41) . for i = 1,2, let p l (nh ) and r l (nh) be e-optimizers for Yf h and X l nh 
respectively, and define 

Phi*) '■= P'idt/h] + !)M an d r l h (t) := f l (([t/h\ + 1 )h). 

By Lemma [3. 2 [ we choose h small enough with r(h) < e/3, such that for * = 1,2, and any po, tq £ T, 
\X i po -E po [U i (p 0 ,4(p 0 ))]\<2e and \Yf 0 - E ro [[/^(to), r 0 )] | < 2e. 
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For t £ [0, oo], let us define 

Wt 1 := Et [U\t,r 2 (t))] and W 2 := E* [U 2 (pl(t), t)\ . 

Lemma 4.1. For i = 1,2, the process W l = {Wf )te[o,ool right continuous. 

Proof. Fix t £ [0, oo). Let t n \t with \t n — 1\ < (1/n) A ([t/M] + 1 )h — t). Obviously, r|(t n ) = T^(t). 
Then 


K-W 7 * 1 ! < |Et n [U(t n ,r 2 (t))\ -E* n [t/(t,r 2 (t))]| + |E t „ [U(t, r 2 (t))] - E f [U(t, r 2 (t))\ \ . 

On the right-hand-side of the inequality above, denote the first term as I and the second term as 
J. We have that 

I <E tn \U{t n ,T%(t))-U{t,T%(t))\ < r(l/n) —»■ 0. 

Now consider J. Observe that the process 

(/. ; =E t+ ,[f/(t,T k 2 (*))]), £M 

is a martingale w.r.t. the filtration (J r i +s ) se [ 0iOO ] satisfying the usual conditions. By [J Theorem 
3.13, page 16], (/ s ) s e[ 0 ,oo] is right continuous. Hence, J —> 0. □ 

Consider the sub-Dynkin game G] 


sup inf Ej 1 {po<ro} + ^ro^{po>'ro} “h Z po 1 {po=to}\ 

PoeTt T o£Tt 

= inf sup E t [^p 0 ^{po< T o} + ^r 0 -*-{po>'ro> + ^po^{pO=ro}\ , 
T 0 eTt P0 (zT t 


(4.1) 


and the game G 2 


v , : = 


inf sup E t [X p 0 l{p 0< r 0 } + ^to^{po>tq} + ^po^1po=to}] 
POG U T 0 £Tt 

sup inf Ej [Np 0 l{p 0< r 0 } + Y T0 l{p 0 >T 0 } + Z l {po=To} \ ■ 

roeTt PoG/t 


(4.2) 


By 13 Lemma 8], the process (r’Dtgp.oo] i s right continuous for i = 1,2. Define stopping times 
pi \= inf{f > 0 : v} < W f ] + e} and M 2 '■= inf{t > 0 : v 2 < W 2 + e}. 

Since X 1 < W 1 , we have that 


pi < inf{£ > 0 : vl < X\ + e}. 

Therefore, the process is a sub-martingale. Similarly, (u 2 ) te [ 0>At2 ] is a sub-martingale. 

Let 5 > 0 that will be chosen later on. For * = 1,2, let (p 1 +s , T p i+ s) *= 7^. +(5 be an £ - sa ddle point 
for game G l +s defined in (14.11) and (14.21) . Define 


II 

* o 

I Mi, 

[pL+s* 

if Mi < M2, 

if mi > M2, 

ri(t) - | 

kco, 

[pl(t), 

if t > pi A M 2 + <5 and mi > M 2 

otherwise, 

H 

\ T pi+5’ 
[fJ-2, 

if Mi < M2, 

if mi > M2, 

n rw ~ | 

Ikw. 

[r 2 h (t), 

if t > Mi A M 2 + 5 and Mi < M2 

otherwise. 
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Let p* := (Pq,P*) and r* := (tq,t*). It can been shown that p*,T* € X. 

Proposition 4.2. For 5 small enough, (p*,r*) is an 18 e-Nash equilibrium for the non-zero-sum 
game m- Therefore, Theorem \2.f \ holds. 

Proof. 


u\p*,t*) = IE \U l (p*[r*],T*[p*])] 

= E [i U 1 (p* [r*], r* [r*]) (l {Atl < M2} + l {w>/i2} )] 

= E [lJ 1 (mi,t|(/xi)) 1{ M1 < M2 } + t/ 1 (Pft(/^2),M2) l{ Ml >/, 2 }] 

= E [E w [I/ 1 (mi,^(w))] l {Atl <^ 2} +E^ 2 [t/ 1 (^(^ 2 )^ 2 )] 1 {/il>jU 2 }] 

— E \Win l{pi<P2> + -*-{mi>M 2}] — e - 

Now for any p = (po,Pi) G X, we consider u 1 (/9, r*) = E[LI 1 (p[t*],t*[p])]. We discuss five cases. 
Case 1: A± := {po < pi A ^ 2 }. We have that 

= E [U 1 (po,T%(po)) l^i] 

= E[<Uj 

— E [^poA/uA/^ ^4i] 

— E [PpoAp,iAp 2 [XuA/pJ ^4i] 

= E KiAp 2 U 1 ] 

“ E [('Xu -*-{pi<P2} + l ’p2^{Ml>P2}) ] 

< E 1{ M1 <^ 2 ) + i p 1 2 1 {pi>p 2 }) 1 m] +e. 

Case 2: A 2 := {p\ A P 2 < Po < Pi A P 2 + 5} fl {pi < ^ 2 }- By Lemma IXT1 

E \Ki+s- w h\^ 0 ’ 

and 

E {([/ii//i] + l)/i — < 2d} —> 0, 5 0 . 

We choose 6 small enough, such that r(6) < e, and 

E|Wi +5 -W Ml | <e 

and 


P{([pi/h] + l)/i - Pi < 25} < e/M, 
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where M > 0 is a constant such that \U 1 \ < M uniformly in (s,t, u). Then 


E {U l {p[r*U*\p])lA 2 \ = 

< 

< 

< 


E [U 1 (po,7ft(po)) 1 Aa] 

® [ U 1 (po, i + 5)) 1 j 4 2 ] + 2e 

E [t / 1 (pi + <5, (pi + (5)) 1a 2 ] + 3e 
E + 3e 
EKU 2 ]+4e 

E [(Kh^} + y M 1 2 1 {m>M2}) ^ 2 ] + 4e - 


Case 3: A 3 := {pi A P 2 < po < Pi A P 2 + 4 } H {pi > ^ 2 }- We choose 5 small enough, such that 
r(S) < e and 

E K + s- y ,l,\<‘- 

Then we have that 


E ^ 1 (p[r*],T*[p])l A3 ] 


= E [(U 1 (P2,P2) l{p 0 =At 2 } + ^ (^l(^2),P2) 1 {p 0 >M2}) !a 3 ] 

< E [( 4 M2 1 {po=At2} + U 1 (pi(p2) + <^,P2 + <j) l{po>At 2 }) l- 4 3 ] + 2 e 

— E [( E /i 2 ^{P 0 =M 2 l + 4 P2+5-*-{P0>P2}) ^a 3 ] + 2e 

- E K 2 1 A 3 ]+ 3 e 

= E [(W^ l{ M 1 <p 2 } + 4 p 2 1 {pi>P2}) ^s] + 3e - 


Case 4: A 4 := {po > pi A p .2 + 5} fl {pi < /pj}- We choose 5 small enough, such that 


E Ki+s- v ii I < e - 

By Remark 13.41 we have that 

E [U 1 (p[r*], r*[p]) 1 a 4 ] = E [E M1+5 [f/ 1 (p[r*],r*[p])] U 4 ] 

— E [ t '/'ti+<5 1 A 4 ] + 4e 

< E [v 1 ^ l Ai \ + 5e 

< U 4 ]+66 

= E + ^ 1 2 l{ w > / i 2 }) 1 a 4 ] + 6e - 

Case 5: A 5 := {po > pi A P 2 + <5} fl {/ii > po}. We choose 5 small enough, such that r(<5) < e, 
and 

E |r; 2+i -r;j< e . 
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Then 


E [U l {p[T*],T*\p\)l M \ = 

< 

< 

< 


E [U 1 (pi(h 2 ),I^2) 1a 5 ] 

E [U 1 (pi(p 2 ) + < 5 > P 2 + < 5 ) 1 a 5 \ + 2e 

E K+^As] + 2e 
E + 3e 

E + Y /^2 1 {Ml>A t 2}) ^ 5 ] + 3e - 


From Cases 1-5, we have that 

v}(p, t*) < i^ip*, t*) + 18e, 

for any p € T by choosing 5 small enough. Similar we can show that 

u 2 (/ 9 *, r) < u 2 (p*, t*) + 18e, 
for any r G T by choosing (5 small enough. 


□ 
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